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1. INTRODUCTION 



Corresponding to any time series - 0,i,2. • } is the time series 

{X , ; j = 0,1,2, • • • }, where 

(i) 

/i-l 

is the j th batch mean of size 6 . The batch-means process, or aggregated time series, is 
of interest when observations are actually batched (see, e.g., Telser [1967], Amemiya 
and Wu [1972], and Tiao [1972]), when calculations need to be simplified (see. e.g.. 
Blackman and Tukey [1958, Sec. B. 17]), or when the process mean, E(A), is to be 
estimated. The third context motivates our work. 



n 

Consider estimating E(A) with the average of n observations. A = ]T]A r f n. Using 



batch means to estimate the variance of the sample mean. V (A ), has long been con- 
sidered (see, e.g., Conway [1963]). Brillinger (1973) shows that if the values of a process 
at a distance from each other are only weakly dependent, then the batch means are 
asymptotically independent and normally distributed. Thus, for large batch size 6 , 



V (X ) can be estimated using S 2 k / k, where k = [n / 6], S 2 k = (k - 1)’ 




kX 2 



and [ J 



; = i 

denotes the floor function. (Moran [1965] discusses related estimators.) A nominal 
100(1 — a) percent confidence interval on £(A r ) is then X ±t l _ Q/2k _ l S k k ~ l/ 2 . where 
*i-a/ 2 ,* -i is the 1 - ( a / 2) quantile of Student’s t distribution with k - 1 degrees of freedom. 



The batch means algorithms developed by Mechanic and McKay (1966), Fishman 
(1978), Law and Carson (1979), Schriber and Andrews (1979), and Adam (1983) empiri- 
cally calculate measures of batch dependency for various batch sizes 6 in an attempt to 
determine a reasonably small value of b that yields batch means that are almost 
independent and normally distributed. These procedures require substantial calculation; 
Law and Carson (1979), for example, calculate first-order correlations based on 400 
batches. That so many batches are required for accurate estimation of dependency 
measures is unfortunate, since Schmeiser (1982) shows that, for fixed n. additional 
batches beyond some small number (ten to thirty) do little to improve the statistical 
properties of the batch means confidence interval procedures. The results of Section 2 
are motivated by the idea that knowledge of the relationship between {X, } and {A, } can 
be used to measure properties of { A ; } even for small values of k . 



A second reason for studying the relationship between {A,} and {A ; } is to allow 
more efficient simulation studies of batch-means procedures. Studying the performance 
of several batch-means procedures in the context of various distributions assumptions 
for {A,} requires a large computational effort, especially when the large sample sizes 
required to simulate a system and the large number of replications required for meaning- 
ful conclusions are considered. A crude Monte Carlo method is to generate 
A'^Ao, • • • ,A n and calculate the batch means A,, An. ,A‘| n ,*j for all values of b of 
interest. A computationally more efficient alternative is to derive the properties of {A' ; } 
from the properties of {A,} and to generate directly the batch means {A ; }. as discussed 
in Section 4. 
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A third motivation is that direct insights might result from studying properties of 
{Xj } as functions of the parameters of {X, }. Particularly interesting is the sensitivity of 
{Xj} to the underlying process and to the batch size 6, as discussed in Kang (1984, 
Chapter 5). 

Section 2 contains results relating batch-means processes to arbitrary stationary 
autoregressive moving-average (ARMA) processes. Section 3 considers the special case 
of the underlying process being ARMA (1,1). Section 4 is a summary containing an 
algorithm for determining the batch-means process from the underlying process. 



2. BATCH MEANS OF STATIONARY ARMA PROCESSES 



The ARMA (p ,<?) process {X t } by definition satisfies 

= (2) 
h = 0 h =0 

where <t> 0 = 1 , 0 0 = 1 , and the error terms e, are independent with zero mean and vari- 
ance a ( 2 . The main result of this paper is Theorem 1, which states that batch- means 
processes arising from ARMA underlying processes are themselves ARMA and specifies 
the parameter values. 

Theorem 1. Consider the stationary ARMA(p ,q) process of equation (2). 

Then {X } } is the stai onary ARMA(p ,q) process 

~ S ^ h € j-h 
k =0 h = 0 

where $ 0 =l , <9 0 = 1, the batch-means error terms are independent random 
variables with zero mean and variance a 2 r , and q, <f> u <f> 2 , • • • , and 

0 l .O o, * * • ,0 - are functions (of the parameters of the underlying process and 
the batch size b) given in Lemmas i, 2, and 4 7 respectively. 

The proof of Theorem 1 requires the following lemmas. 

Lemma 1 (Anderson [1979a, p. 155]). If the underlying process {X, } is a 
stationary ARMA(p .q) process , then the batch-means process {X ; } is a sta- 
tionary ARMA(p ,q) process , where q - p -[(p-q)/ &J. 



Anderson uses the more complicated, but equivalent, expression q - 
Lemma 1 has several direct implications, as discussed in the Appendix. 



{p+l)(b-l) 

b 



Lemma 2 (Amemiya and Wu [1972]). The AR parameters 0,,<z> 2 , ■ ■ • .o p 
of the batch means process {X ; } are the coefficients of B\B\ • • * ,B P of 

p 

~\(\-a h b B), respectively , where a,.a 2 , • • * ,a p are the roots of the charac- 

h = 1 

p 

teristic equation ^{B) = h - 0. 

h 0 
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Lemma 3. For any stationary process {A, }, the lag-h autocorrelation of 
the batch-means process {X } } is 



Ph = Corr (X ; , X,+ h ) = 



b b - 1 

Yj l P{h-l)b+t + E 1 P{h-\)b + 26-» 

i=l i=l 



be 



6-1 

where c = 1 + 2 £ (1- (A / 6 ))p A , Ph 

A = 1 

R h = E[(X,--EX)(X i+A -EX)] for h = 0,1,2, • • • and % 



— Rh / Rq> 
= 0 , 1 , 2 , 



and 



Proof. For any stationary process, the lag -h covariance of the batch 
means process is 

Cov (X } , X J+h ) = b ~ 2 SECov(Y (l . 1)t+M Y 0+h _ 1)HA ) (3) 

• =1A =1 

(see. e.g., Kleijnen [1975, p. 507]). which is a special case of the covariance 
of linear combinations of random variables as discussed in Box and Jenkins 
(1976, pp. 28-29). Also, for any stationary process, each batch mean X } 
has variance 

R 0 =V(X } )= c RJ b (4) 

(see, e.g., Fishman [1973, p. 281]). The definition of correlation 
Ph 5Cov(X,-J i+ *)/ [V(A ; )V(X ; ^)] 1/2 
and equations (3) and (4) yield 

Ph = &’ 2 E E Cov [X u . l)k+ti X {}+h _ l)b+k \ / \c R 0 / b] 

■ = i* = i 
6 6 

= E E Corr i^(j '1)6 + . J A (; +h-l)b + * ] / [ be}. 

i = 1 k = 1 

Counting like terms arising from stationarity yields the result, a 

Lemma 4 (Anderson [1971, p. 237]). Consider a stationary ARMA(p ,q) 
process with known AR (autoregressive) parameters ‘ ,<t> P »' variance 

R 0 , and autocorrelation coefficients p x ,p 2 , • • * } p _ . Then the MA ( moving- 
average) parameters 6 x ,9 2 , * • • (0 q are determined. 

Theorem 1 can now be proven using the four lemmas. 



Proof of Theorem 1. 

1. From Lemma 1. the AR and MA orders of { A r ; } are determined: in 
particular, {X ; } is an ARMA(p ) process. 

2. From Lemma 2. the AR parameters of {X ; } are determined. 

3. The autocorrelations • • >P h { q + \)-\ °f a stationary ARMA process 

can be calculated using the algorithm of Sweet and Mazaheri (1979). 
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4. Given the autocorrelations from Step 3, the batch- means variance R 0 
is determined by equation (4) and the batch-means autocorrelations 
p ,,p 2 , • • • ,Pj are determined by Lemma 3. 

5. The MA parameters 0,,0 2) • • • ,9- are then determined from Lemma 
4. □ 

The representation of the batch-means process is not unique. The batch-means 
MA parameters of Lemma 4 require the solution of a polynomial equation of order 2 q to 
determine 0 2 , ■ ■ ■ ,6 - . The 2 q roots can be partitioned V ways into two sets 
(x,,x 2 , ■ • • ,x-) and (x f+1 ,Zj. +2 , • • • ,x 2? ) such that x- +i = 1/ x, . All such subsets of size q 
from (x,,x 2) • • • ,x 2 -) determine 6 u 0 2 , • • • ,6 - corresponding to stochastically equivalent 
processes. But there is a unique subset of size q having | x,- | $ 1, and therefore 
x- +( I = I 1/ x , I ^ 1, for i = 1,2, ••,?. Thus for the ARMA(p ,7) process there exist 
2* — 1 non-invertible processes corresponding to a unique invertible process. 



3. ARMA(l,l) BATCH MEANS 



Now consider the special case of stationary first-order ARMA processes 

A, - 1 - <t> X = e ,• + 0«,-i for i - 1,2, ■ • (5) 

The low-order moments of {A, } are the zero mean, variance 

R*= °< 2 ( 1 + 8 2 — 2<t>9) / (1 -4> 2 ), (6) 

the lag-one autocorrelation 

Px = (1 - <t>0 )(6 - <t>) / (1 + d 2 - 2<p0 ) . (7) 

and lag-h autocorrelations 

Ph = (~4>)ph - 1 = {-<t>) h ~ l p\ for h = 2,3, • • * . (8) 

Closed-form expressions for the parameters of the batch-means process are given in 
Theorem 2. 

Theorem 2. Consider the stationary ARMA(l.l) process of equation (5). 

The corresponding batch- means process is the stationary ARMA(l,l) pro- 
cess 



X } ~i~ <p X j — e ; -i- 8 e } ^ x for j - 1,2, (9) 

where 

(-1 )* + V (10) 

(2 op, + r- 1) - !(l-^ 2 )(l-(2 Px + 0) 2 V 1 ' 2 Ml x 

a - (111 

2(p, -+- o) 



and 
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where 



( 12 ) 



a r 2 - R 0 (l-f 2 ) / (P - 2$9 + 1) 

«o= c fl 0 / 6 (13) 

Fi = />i(l+^) 2 / [6c (1+^) 2 ] (14) 

and 

C = 1 + {2/» 1 |(-^)‘ + 6 (1 + *) - lj / [6 (1+^) 2 ]} (15) 

Proof of Theorem 2. 

Equations (13), (14), and (15) follow from Lemma 3 and equation (4) 
via equation (8). Since {.V,} is ARMA(l,l), {A,} is also ARMA(1,1) by 
Lemma 1: that is. equation (9) holds. Equation (10) is a special case of 
Lemma 2. Since {.Y ; } is ARMA(l,l). equation (7) yields 

p , = (i-$0)(F-w) / [1+P-2&F). 

Solving for 9 yields the two roots of equation (ll). Since {A,-} is an 
ARMA(l,l) process, equation (6) holds with the batch-means parameters: 

R 0 = o r 2 (l+P-U9) / (l-^ 2 ) . 

Solving for the variance of the batch-means error term, a T 2 , with either 
value of 9 from equation (ll) yields equation (12). (But note that the 
value of o — 2 depends on the choice of 9 ). □ 



4. SUMMARY 

Properties of batch means are studied under the assumption that the underlying 
process is ARMA(p,$). For ARMA( 1 , 1 ) processes, closed-form expressions for the 
corresponding batch-means processes are obtained. A numerical procedure is developed 
for calculating the parameters of the ARMA batch-means process from the parameters 
of the underlying process and the batch size 6 . This procedure is stated concisely here 
for convenience. 

ARMA(p.q) Procedure. 

Given parameters < 6 ,,<j!> 2 . • • • . 9 u 9 2 , • • ,9 q . error variance a,-, and batch 

size 6 , calculate 

1- 4 = P ~ L(p -9 )/ 6| 

2 . • • • ,? p using Lemma 2 

3. /£ 0 ,Pi,P 2 , ■ ' • , from the Yule-Walker equations, probably using 

the algorithm of Sweet and Mazaheri (1979) 

4. c = 1 -t- 2 £(1 (6 b)) Pk 

h I 
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5. R o from equation (4) 

6. P\,Ji, ' • • ,Pj using Lemma 3 

7. 6 i,0 2 , ■ ■ ■ ,9 - from Lemma 4 

8. <7 r 2 from equation (13) of Anderson (1971, p. 237). 

A FORTRAN implementation of the ARMA(p,?) procedure is given in Kang (1984). 

When the underlying process is ARMA(l,l), the following closed-form procedure 
can be used: 

ARMA(l,l) Procedure. 

Given parameters <j>, 9 , error variance <r ( 2 , and batch size 6 , calculate 

1 . q = 1 

2. $ from equation (10) 

3. R o from equation (6), p x from equation (7) 

4. c from equation (15) 

5. R 0 from equation (13) 

6. p } from equation (14) 

7. 9 from equation (11) 

8. <7 r 2 from equation (12) 

Notice in the ARMA(l,l) special case that calculation in step 3 of all 26-1 autocorrela- 
tions of the underlying process is not necessary. 

If the underlying error terms t , are normally distributed, then the batch-means 
error terms t , are also normally distributed (see, e.g., Johnson and Kotz [1971, p. 5 1] ) . 
Therefore, generation of random variates directly from the batch-means process is 
straightforward using equation (9), thereby avoiding the costly computations of aggre- 
gating observations from the underlying process. Initialization for steady-state results is 
straightforward for AR and MA processes, but initialization for ARMA is complicated 
unless the process is warmed-up by discarding some initial observations (Anderson 
[1979b]). 
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APPENDIX 



Although Lemma 1 is simple to state compactly, its implications are more clear 
when cases are considered individually: 



If 


and 


then 




b <p - q 


i 

a. 

V/ 

1 

V/ 


P >7 


b-p-q 


q = p - 1 




b>p-q 


q = p 


p = q 




q=p 




b <q -p 


p +2^ q ^q 


p <q 


b — q — p 


II 

"O 

+ 




b>q-p 


7 = P -1 



Many results can be stated immediately from examination of these individual 
cases. Five such results are: 

Result 1. If {X, } is AR{p), then {X } } is ARMA(p ,q), as studied by 

Amemiya and Wu (1972). Additionally , l < q ^ p . 

Result 2. If {X t } is MA(q), then {X } } is MA(q ), where 1 ^ q ^ q. 

Result 3. If {X t } is AR or ARMA with batch size satisfying 0 ^ p -q < 6 , 

then {Xj } is ARMA(p ,p ). 

Result 4. If p ^ q , then lim q = p . 

b -* oc 

Result 5. If p < q f then lim q = p-fi. 

b -00 

Of course, considering only the order of the batch-means process can be mislead- 
ing. For example, Results 4 and 5 indicate that large batches lead to MA components 
of order p or p+l; in particular, a batched MA(?| process converges to an MA(1) pro- 
cess. But large batches are asymptotically independent. The explanation is that 0 ] is 
approaching zero as batch size increases. An implication is that, even for this nicest 
case of ARMA underlying processes, estimation of the order of the batch-means process 
is likely to be difficult. 
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